Facial authentication device, facial authentication method, and program recording medium

ABSTRACT

This facial authentication device is provided with: a detecting means for detecting a plurality of facial feature point candidates, using a plurality of different techniques, for at least one facial feature point of a target face, from a plurality of facial images containing the target face; a reliability calculating means for calculating a reliability of each facial image, from statistical information obtained on the basis of the plurality of detected facial feature point candidates; and a selecting means for selecting a facial image to be used for authentication of the target face, from among the plurality of facial images, on the basis of the calculated reliabilities.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 17/529,479 filed on Nov. 18, 2021, which is acontinuation application of U.S. patent application Ser. No. 16/623,478filed on Dec. 17, 2019, which issued as U.S. Pat. No. 11,210,498, whichis a National Stage Entry of international applicationPCT/JP2018/023066, filed Jun. 18, 2018, which claims the benefit ofpriority from Japanese Patent Application 2017-124335 filed on Jun. 26,2017, the disclosures of all of which are incorporated in their entiretyby reference herein.

TECHNICAL FIELD

The present invention relates to a facial authentication device, afacial authentication method, and a program recording medium forperforming facial authentication by using a face image.

BACKGROUND ART

A facial authentication technique of identifying an individual from aface image of a person is used in various scenes including a securityfield. For example, a video (moving image) captured by a surveillancecamera is used in identifying a person from face information extractedfrom the moving image.

At the time of facial authentication using a moving image, when a faceof a person is included in a range of a plurality of frames thatconstitute the moving image, performing facial authentication on all theframes each including the face increases calculation processing for theauthentication, and takes time to acquire an authentication result.Further, a plurality of the frames include a frame including a faceunsuitable for facial authentication, such as a face in which imageblurring, partial covering, or the like exists. Thus, it is desirable toperform facial authentication by selecting, from a moving image, a framesuitable for facial authentication.

In facial authentication, a pre-registered collation face imageincluding a face of a person desired to be identified is collated with aframe (hereinafter, referred to also as “a collation-target face image”)that is selected from a moving image and includes a collation-targetface. In the collation, distinctive feature points (hereinafter,referred to as face feature points) including facial organs such as eyesand a nose, a facial skeletal structure, and the like are detected fromeach of the collation face image and the collation-target face image,and the two images are collated with each other, based on the detectedface feature points. As a result of the collation, it is determinedwhether or not the respective faces included in the collation face imageand the collation-target face image are faces of the same person.

One example of a technique of selecting an image suitable for facialauthentication is disclosed in PTL 1.

-   -   PTL 1 discloses an individual authentication device that detects        an orientation of a face of a target user, performs face        recognition depending on the orientation of the face, and        performs individual authentication.    -   PTL 2 discloses a face recognition device that determines        whether or not a covering object exists, based on a detection        signal from a face-part detection unit, and induces a user to        remove the covering object when the covering object exists.    -   PTL 3 discloses a face recognition device that controls,        depending on a size of a face, a method of transferring face        image data necessary for face recognition processing, and        thereby reduces a transfer amount. PTL 4 discloses a        face-feature-point position correction device that can output a        highly accurate face-feature-point position even when        low-reliability-degree information is input as to one or a        plurality of face feature points.    -   NPL 1 discloses a face recognition method in which a face image        without blinking is selected, and facial authentication is        performed. NPLs 2 and 3 each disclose one example of a        face-feature-point detection method.    -   NPL 4 discloses a technique of accurately extracting an eye area        even in the case of a face image in which a face is tilted.

CITATION LIST Patent Literature

-   -   [PTL 1] Japanese Unexamined Patent Application Publication No.        2002-288670    -   [PTL 2] Japanese Unexamined Patent Application Publication No.        2016-099939    -   [PTL 3] International Publication No. WO 2010/044214    -   [PTL 4] International Publication No. WO 2011/148596 [Non Patent        Literature]    -   [NPL 1] Kunihiko Omori, Kazuto Murakami, “A Simple Method to        Extract the Best Shot from Sequential Images”, The Institute of        Electronics, Information and Communication Engineers, Japan,        Technical Report, HIP, Human Information Processing, 101(423),        Nov. 8, 2001, pages 27 to 32    -   [NPL 2] T. F. Cootes, G. J. Edwards, C. J. Taylor, “Active        appearance models”, IEEE Transactions on pattern analysis and        machine intelligence, Vol. 23, No. 6, June 2001, pages 681 to        685    -   [NPL 3] Xiangxin Zhu, Deva Ramanan, “Face detection, pose        estimation, and landmark localization in the wild”, Computer        Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on        2012    -   [NPL 4] Midori Shinkaku, and two others, “Improvement of Face        Recognition Accuracy Using Eyes Detection by Haar-Like        Features”, [Online],    -   [Searching Date: May 15, 2017], Internet        <https://www.ieice.org/tokyo/gakusei/kenkyuu/14/pdf/80.pdf>

SUMMARY OF INVENTION Technical Problem

As described above, PTL 1 discloses that an image of a face oriented tothe front is selected and collated with a pre-registered face imageoriented to the front, and a person is thereby identified with highaccuracy.

Meanwhile, collation between a collation face image and acollation-target face image performed in facial authentication has aproblem that accuracy of the authentication tends to decline due topositional deviation of detected face feature points.

Herein, positional deviation refers to a state where face feature pointssuch as eyes and a nose are detected at positions deviated frompositions thereof in the face image. For example, when a face featurepoint indicating a pupil center of a right eye is detected in a certainface image, a point on a left side of or a point on a right side of thepupil of the right eye is detected as a face feature point in somecases. A state where a face feature point is thus detected at a positiondeviated from a detection-desired position such as a pupil center of aright eye is referred to as positional deviation. The positionaldeviation is caused by a lot of noise included in a face image, a smallsize of a face included therein, a covering over a face, or the like,for example.

When the above-described positional deviation occurs, face images arecollated with each other in a state where face feature points to becompared are detected at positions deviated from each other between acollation face image and a collation-target face image. For this reason,there is a problem that accuracy of facial authentication declines.

The technique disclosed in PTL 1 can prevent a combination of a sideface and a front face from being used in facial authentication, butcannot prevent a decline in accuracy of facial authentication beingcaused by positional deviation of a detected face feature point.

PTLs 2 to 4 and NPLs 1 to 4 also do not disclose prevention of a declinein accuracy of facial authentication being caused by positionaldeviation of a detected face feature point.

The present invention has been made in view of the above-describedproblem, and mainly aims to provide a facial authentication device andthe like that can suppress influence of positional deviation of adetected face feature point and achieve highly accurate facialauthentication.

Solution to Problem

An aspect of the present invention is a facial authentication device.The facial authentication device includes detection means for detecting,from each of a plurality of face images including a target face, aplurality of face-feature-point candidates for at least one face featurepoint of the target face, by using a plurality of different methods;reliability degree calculation means for calculating a reliabilitydegree of each of face images, from statistical information acquiredbased on the plurality of detected face-feature-point candidates; andselection means for selecting, based on the calculated reliabilitydegrees, from the plurality of face images, a face image to be used inauthentication of the target face.

An aspect of the present invention is a facial authentication method.The facial authentication method includes detecting, from each of aplurality of face images including a target face, a plurality offace-feature-point candidates for at least one face feature point of thetarget face, by using a plurality of different methods; calculating areliability degree of each of face images, from statistical informationacquired based on the plurality of detected face-feature-pointcandidates; and selecting, based on the calculated reliability degrees,from the plurality of face images, a face image to be used inauthentication of the target face.

An aspect of the present invention is a program recording medium. Theprogram recording medium records a program causing a computer toexecute: processing of detecting, from each of a plurality of faceimages including a target face, a plurality of face-feature-pointcandidates for at least one face feature point of the target face, byusing a plurality of different methods; processing of calculating areliability degree of each of face images, from statistical informationacquired based on the plurality of detected face-feature-pointcandidates; and processing of selecting, based on the calculatedreliability degrees, from the plurality of face images, a face image tobe used in authentication of the target face.

Advantageous Effects of Invention

According to the present invention, it is possible to attain anadvantageous effect that influence of positional deviation of a detectedface feature point can be suppressed, and highly accurate facialauthentication can be achieved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a facialauthentication device according to a first example embodiment of thepresent invention.

FIG. 2 is a block diagram illustrating a configuration of a facialauthentication device according to a second example embodiment of thepresent invention.

FIG. 3 is a flowchart that illustrates processing performed by aselection unit of the facial authentication device according to thesecond example embodiment of the present invention.

FIG. 4 is a diagram illustrating one example of a face area included ina face image acquired by a face-feature-point candidate detection unitof the facial authentication device according to the second exampleembodiment of the present invention.

FIG. 5 is a diagram illustrating an example of face feature points asdetection targets of the face-feature-point candidate detection unit ofthe facial authentication device according to the second exampleembodiment of the present invention.

FIG. 6A is a diagram illustrating an example of face-feature-pointcandidates detected by the face-feature-point candidate detection unitof the facial authentication device according to the second exampleembodiment of the present invention.

FIG. 6B is a diagram illustrating an example of face-feature-pointcandidates detected by the face-feature-point candidate detection unitof the facial authentication device according to the second exampleembodiment of the present invention.

FIG. 6C is a diagram illustrating an example of face-feature-pointcandidates detected by the face-feature-point candidate detection unitof the facial authentication device according to the second exampleembodiment of the present invention.

FIG. 7 is a diagram illustrating one example of an eye area whereface-feature-point candidates are detected by the face-feature-pointcandidate detection unit of the facial authentication device accordingto the second example embodiment of the present invention.

FIG. 8 is a flowchart illustrating processing performed by anauthentication unit of the facial authentication device according to thesecond example embodiment of the present invention.

FIG. 9 is a diagram illustrating one example of coordinate values offace-feature-point candidates calculated by an integratedface-feature-point calculation unit of the facial authentication deviceaccording to the second example embodiment of the present invention.

FIG. 10 is a diagram illustrating one example of a hardwareconfiguration of a computer device that implements the facialauthentication device of each of the example embodiments.

EXAMPLE EMBODIMENT

Hereinafter, example embodiments of the present invention are describedin detail with reference to the drawings.

First Example Embodiment

FIG. 1 is a block diagram illustrating a configuration of a facialauthentication device 100 according to a first example embodiment of thepresent invention. As illustrated in FIG. 1 , the facial authenticationdevice 100 includes a detection unit 110, a reliability degreecalculation unit 120, and a selection unit 130.

The detection unit 110 detects a plurality of face-feature-pointcandidates for at least one face feature point of a target face, fromeach of a plurality of face images each including the target face, byusing a plurality of different methods. The reliability degreecalculation unit 120 calculates a reliability degree of each of the faceimages from statistical information acquired based on a plurality of thedetected face-feature-point candidates. Based on the calculatedreliability degrees, the selection unit 130 selects from a plurality ofthe face images the face image used in authentication of the targetface.

Note that the detection unit 110, the reliability degree calculationunit 120, and the selection unit 130 are implemented by aface-feature-point candidate detection unit 231, a reliability degreecalculation unit 232, and a face image selection unit 233 respectivelyas one example which are described in the following example embodiment.

According to the first example embodiment, a face image in which a facefeature point is detected with high accuracy is selected from aplurality of face images, and the selected face image is used inauthentication, thus attaining an advantageous effect that influence ofpositional deviation of a detected face feature point can be suppressed,and highly accurate facial authentication can be achieved.

Second Example Embodiment

FIG. 2 is a block diagram illustrating a configuration of a facialauthentication device 200 according to a second example embodiment ofthe present invention. As illustrated in FIG. 2 , the facialauthentication device 200 includes an input unit 210, a selection unit230, an authentication unit 250, and an output unit 270.

The selection unit 230 includes a face-feature-point candidate detectionunit 231, a reliability degree calculation unit 232, and a face imageselection unit 233. The authentication unit 250 includes an integratedface feature point calculation unit 251, a normalization unit 252, acollation unit 253, and a template storage unit 254.

The input unit 210 acquires video (moving image) data generated by amonitoring camera or the like. The video data include a plurality offrames (still images) each including a face of a person.

The selection unit 230 has a function of selecting a frame used incollation for the below-described facial authentication, from aplurality of the consecutive frames constituting the video data acquiredby the input unit 210.

The authentication unit 250 has a function of performing the facialauthentication on the target face, based on the frame selected by theselection unit 230. The output unit 270 outputs a result of theauthentication performed by the authentication unit 250.

The following describes an outline of each constituent element of theselection unit 230.

The face-feature-point candidate detection unit 231 acquires, from videodata acquired by the input unit 210, face images that are frames eachincluding a face, and detects candidates (hereinafter, referred to alsoas “face-feature-point candidates”) of a face feature point from each ofthe acquired face images, by using a plurality of differentface-feature-point detection methods.

The face feature point is defined based on information such as an organand a skeletal structure of a face. For example, a mouth corner, a pupilcenter, and the like are used as detection-target face feature points.The face-feature-point candidate detection unit 231 detects a pluralityof face-feature-point candidates for each of the detection-target facefeature points, using a plurality of different face-feature-pointdetection methods.

The reliability degree calculation unit 232 calculates a reliabilitydegree of each of the face images, based on statistical information of aplurality of the face-feature-point candidates detected by theface-feature-point candidate detection unit 231 (details are describedbelow). An example used as the statistical information is variance ofrespective positions (coordinate values) of the face-feature-pointcandidates.

The face image selection unit 233 selects the face image suitable forfacial authentication, based on the reliability degrees of therespective face images calculated by the reliability degree calculationunit 232. The face image selection unit 233 may select a face imagewhose reliability degree is the maximum value, or may select the faceimage whose reliability degree is equal to or larger than a thresholdvalue.

In the above-described procedure, a face image suitable for the facialauthentication is selected.

Next, an outline of each constituent element of the authentication unit250 is described.

For a face image selected by the face image selection unit 233, theintegrated face-feature-point calculation unit 251 calculates, based ona plurality of face-feature-point candidates detected for each ofdetection-target face feature points, an integrated face feature pointused in collation for facial authentication. The integratedface-feature-point calculation unit 251 may calculate the integratedface feature point, for example, based on an average of positions(coordinate values) of the respective face-feature-point candidates.

The normalization unit 252 corrects and normalize a position and anorientation of the face, based on the face image selected by the faceimage selection unit 233 and the integrated face feature pointscalculated by the integrated face-feature-point calculation unit 251. Inthe calculation processing for the normalization, well-knownnormalization device and normalization technique can be used.

The collation unit 253 generates a face collation template from theimage (hereinafter, referred to also as “the normalized image”)normalized by the normalization unit 252, and collates the generatedface collation template with a template stored in the template storageunit 254, thereby performing facial authentication, and identifying aperson included in video data.

The template storage unit 254 stores the face collation template of aperson desired to be identified.

The output unit 270 outputs as a result of the facial authentication theperson identified by the collation unit 253. For example, the outputunit 270 may output the identified person to a display.

FIG. 3 is a flowchart that illustrates processing in which the selectionunit 230 of the facial authentication device 200 according to the secondexample embodiment selects from video data a face image suitable forfacial authentication. With reference to FIG. 3 , the followingdescribes the processing performed by the selection unit 230.

The input unit 210 in FIG. 2 acquires video data from an externalmonitoring camera or the like. From the video data acquired by the inputunit 210, the face-feature-point candidate detection unit 231 acquires aface image that is a frame including a face (step S210). As to theacquired face image, the face-feature-point candidate detection unit 231detects a plurality of face-feature-point candidates for each ofdetection-target face feature points, by using a plurality of differentface-feature-point detection methods (step S220).

FIG. 4 is an enlarged view illustrating an area (hereinafter, referredto also as “a face area”) that includes a face and that is included inthe face image acquired by the face-feature-point candidate detectionunit 231. FIG. 5 is a diagram illustrating an example of face featurepoints that are detection targets of the face-feature-point candidatedetection unit 231. In the example illustrated in FIG. 5 , thedetection-target face feature points are indicated by the marks “x”. Theexample illustrated in FIG. 5 indicates that five points of a pupilcenter of a right eye, a pupil center of a left eye, a top portion of anose, a right mouth corner, and a left mouth corner are thedetection-target face feature points.

By using a plurality of different face-feature-point detection methods,the face-feature-point candidate detection unit 231 detects a pluralityof face-feature-point candidates for each of the above-described fivepoints, for example. Assuming that the number of a plurality of thedifferent face-feature-point detection methods used by theface-feature-point candidate detection unit 231 is n (n is an integerequal to or larger than two), the face-feature-point candidates aredefined as a set of n face feature points.

FIG. 6A to FIG. 6C are diagrams illustrating respective examples offace-feature-point candidates detected in respective face areas includedin three face images acquired from video data. For example,face-feature-point candidates may be detected for each of face areas ofa certain person in three consecutive face images that are included invideo data and each of which includes a face of the person, but there isno limitation to this.

FIG. 6A to FIG. 6C illustrate the examples in which for each of the faceareas included in the three face images, the face-feature-pointcandidate detection unit 231 detects three face-feature-point candidatesfor each of the above-described five points (a pupil center of a righteye, a pupil center of a left eye, a top portion of a nose, a rightmouth corner, and a left mouth corner), by using three respectivedifferent face-feature-point detection methods.

Herein, examples used as the face-feature-point detection methods mayinclude the face-feature-point detection method described in NPL 2 orNPL 3. These methods are examples, and other methods may be used.

Alternatively, a well-known face-feature-point detection method in whichdifferent data sets are learned may be used as a plurality of differentface-feature-point detection methods. For example, data set is randomlydivided into two sets, and each of the data sets is learned by theface-feature-point detection method described in NPL 2, for example. Thethus-acquired two kinds of models may be used as two different kinds offace-feature-point detection methods.

Alternatively, a well-known face-feature-point detection method in whichrespective data sets represented by respective different featurequantities are learned may be used as a plurality of differentface-feature-point detection methods. For example, a data setrepresented by RGB values of a color image and a data set represented bygray-scale converted luminance values are learned by theface-feature-point detection method described in NPL 2, for example. Thethus-acquired two kinds of models may be used as two different kinds offace-feature-point detection methods.

Subsequently, the reliability degree calculation unit 232 calculates areliability degree of each of the face images, based on statisticalinformation of a plurality of face-feature-point candidates detected bythe face-feature-point candidate detection unit 231 as described above(step S230).

The reliability degree calculation unit 232 may use, as the statisticalinformation, variance of respective positions of the face-feature-pointcandidates. Specifically, the reliability degree calculation unit 232calculates a reliability degree of the face image in the followingprocedure, for example.

FIG. 7 is an enlarged view of an area (hereinafter, referred to also as“an eye area”) that includes an eye and that is included in the facearea where face-feature-point candidates are detected as illustrated inFIG. 6A, for example. FIG. 7 illustrates that three points p, q, and ras face-feature-point candidates of a pupil center in the right eye aredetected.

It is assumed that in the eye area illustrated in FIG. 7 , for example,an xy coordinate system has an x-axis of a horizontal direction of theimage and a y-axis of a vertical direction of the image, and coordinatevalues of the face-feature-point candidates p, q, r are p(x₁, q(x₂, y₂),and r(x₃, y₃) respectively.

In this case, variance v of respective positions of theface-feature-point candidates is calculated by the Equation (1), forexample.

v=(⅓)*((x ₁ −m){circumflex over ( )}2+(y ₁ −n){circumflex over ( )}2+(x₂ −m){circumflex over ( )}2+(y ₂ −n){circumflex over ( )}2+(x ₃−m){circumflex over ( )}2+(y ₃ −n){circumflex over ( )}2)  (1)

where

m=(⅓)*(x ₁ +x ₂ +x ₃)

n=(⅓)*(y ₁ +y ₂ +y ₃)

Note that in the Equation (1) and the following Equations, “/”, “*”, and“{circumflex over ( )}” designate a division, a multiplication, and apower respectively.

By using the Equation (1), the reliability degree calculation unit 232calculates variance v of respective positions of face-feature-pointcandidates also for each of the detected face feature points such as atop portion of a nose and a mouth corner, as in the case of theabove-described pupil center of the right eye.

Subsequently, the reliability degree calculation unit 232 calculates anaverage value of the variance v acquired for the respective detectedface feature points, i.e., average variance u. Here, assuming that thevariance of the respective face-feature-point candidates for the pupilcenter of the right eye, the pupil center of the left eye, the topportion of the nose, the right mouth corner, and the left mouth cornerare v₁, v₂, v₃, v₄, and v₅ respectively, the average variance u iscalculated by the Equation (2).

u=(⅕)*(v ₁ +v ₂ +v ₃ +v ₄ +v ₅)  (2)

Subsequently, the reliability degree calculation unit 232 calculates anegative exponential function of the average variance u by the followingEquation (3), thereby calculating a reliability degree s.

s=exp(−u)  (3)

The reliability degree s takes a value equal to or larger than zero andequal to or smaller than one.

Note that as described above, the reliability degree calculation unit232 uses, as the statistical information, variance of respectivepositions of face-feature-point candidates, and uses a negativeexponential function of average variance in calculating a reliabilitydegree, but there is no limitation to this. For example, the reliabilitydegree calculation unit 232 may use a standard deviation instead ofvariance. Further, the reliability degree calculation unit 232 may use asigmoid function instead of a negative exponential function of averagevariance.

Further, although in the above description, a reliability degree iscalculated based on an average of variance of respective positions offace-feature-point candidates for every detection-target face featurepoint, there is no limitation to this. For example, the reliabilitydegree calculation unit 232 may calculate a reliability degree, based onvariance of respective positions of face-feature-point candidates for atleast one of detection-target face feature points.

Subsequently, the face image selection unit 233 selects a face imagesuitable for facial authentication, based on reliability degreescalculated as described above (step S240). Specifically, the face imageselection unit 233 may select, as a face image suitable for facialauthentication, a face image whose reliability degree calculated by thereliability degree calculation unit 232 is the maximum value. Forexample, from the diagrams illustrated in FIG. 6A to FIG. 6C, the faceimage selection unit 233 may select, as a face image suitable for facialauthentication, FIG. 6C whose reliability degree is the maximum value,i.e., whose variation of the respective positions of theface-feature-point candidates is smallest.

Alternatively, for example, the face image selection unit 233 may selecta face image having a reliability degree equal to or larger than anarbitrary threshold value. In this case, a plurality of face images maybe selected.

As described above, a face image is selected by using a reliabilitydegree based on an average of variance of the face-feature-pointcandidates, and thereby, for example, a face image whose variation ofrespective positions of face-feature-point candidates is large can beexcluded from a face image used in facial authentication.

For example, as to a face image including a face in which a mouth iscovered with a hand, a position of a feature point of the mouth isambiguous, and thus, variance of face-feature-point candidates of themouth increases. Such a face image can be excluded from a face imageused in facial authentication when a reliability degree based on anaverage of variance of respective positions of face-feature-pointcandidates is used as described above.

In the above procedure, the selection unit 230 selects a face imagesuitable for facial authentication.

FIG. 8 is a flowchart illustrating processing in which theauthentication unit 250 of the facial authentication device 200according to the second example embodiment performs facialauthentication, by using a face image selected by the selection unit230.

With reference to FIG. 8 , the following describes the processingperformed by the authentication unit 250.

The integrated face-feature-point calculation unit 251 of theauthentication unit 250 acquires a selected face image from the faceimage selection unit 233 (step S310). For the acquired face image, basedon a plurality of face-feature-point candidates detected for each ofdetection-target face feature points, the integrated face-feature-pointcalculation unit 251 calculates an integrated face feature point used incollation (Step S320). Specifically, for example, the integratedface-feature-point calculation unit 251 calculates an average value ofrespective coordinate values of a plurality of the face-feature-pointcandidates, and sets the average value as a coordinate value of theintegrated face feature point.

FIG. 9 is a diagram illustrating an example in which as to the exampleillustrated in FIG. 7 , an average value of the respective coordinatevalues of a plurality of the face-feature-point candidates concerningthe pupil center of the right eye is calculated as a coordinate value ofan integrated face feature point. As illustrated in FIG. 9 , theintegrated face-feature-point calculation unit 251 calculates theaverage value (m, n) of the respective coordinate values of theface-feature-point candidates p, q, and r, i.e., calculates

m=(⅓)*(x ₁ +x ₂ +x ₃) and

n=(⅓)*(y ₁ +y ₂ +y ₃)

as an x coordinate value and a y coordinate value of the integrated facefeature point, respectively.

Integrating a plurality of face-feature-point candidates in this mannerenables face-feature-point detection to be made more accurately than inthe case of using a face feature point detected by one certain method.

Note that the integrated face-feature-point calculation unit 251 mayset, as a coordinate value of an integrated face feature point, aweighted average value of coordinate values that is acquired by applyingan arbitrary weight to each of the face-feature-point candidates.

Subsequently, the normalization unit 252 corrects a position and anorientation of the face, based on the face image selected by the faceimage selection unit 233 and an integrated face feature point calculatedby the integrated face-feature-point calculation unit 251, and therebynormalizes the face image (step S330). Herein, the normalization refersto processing of adjusting positions of face parts such as eyes and amouth in order to perform face collation between face images.

For example, the normalization unit 252 may use, in the normalization, awell-known normalization method in NPL 4 or the like. For example, thenormalization unit 252 may acquire an in-plane rotation angle of a face,from coordinate values of integrated face feature points of pupilcenters of both eyes, performs reverse rotation such that the right eyeand the left eye become horizontal, and performs normalization into animage having a size of 64×64 pixels.

Note that the above-described normalization method performed by thenormalization unit 252 is one example, and another well-knownnormalization method may be used.

Subsequently, the collation unit 253 calculates a face collationtemplate from an image normalized by the normalization unit 252,collates the calculated template with a face collation template storedin the template storage unit 254, and calculates a face collation score(step S340).

Herein, an arbitrary template may be used as the face collationtemplate. For example, a template using a feature quantity of ahistogram of oriented gradients (HOG) extracted from a normalized image,or a template using a normalized image itself as a feature quantity maybe used. Further, for example, normalized correlation or a Eucliddistance may be used in calculating a score of collation betweentemplates.

The collation unit 253 may calculate a collation score f, by using anormalized image having a size of 64×64 pixels described in NPL 4, forexample. Specifically, the collation unit 253 may calculate a collationscore f as follows, for example. In other words, the collation unit 253uses, as a template, a normalized image that has been rearranged into asingle row of a 4096-dimensional vector by raster scanning, for example.

When two templates t1 and t2 are given, a collation score f usingnormalized correlation can be calculated by the following equation (4):

f=<t1·t2>/(<t1·t1>×<t2·t2>){circumflex over ( )}(½)  (4)

where <t1·t2> designates an inner product of t1 and t2.

The collation score using the normalized correlation becomes the maximumvalue i.e., one when the two templates are the same vector, and thusbecomes a higher value as the templates are more similar to each other.For this reason, the normalized correlation can be used as a facesimilarity degree.

When a face collation score calculated as described above is larger thana threshold value, the collation unit 253 may determine that acollation-target template matches a face collation template stored inthe template storage unit 254. When the face collation template issuccessfully matched, it can be specified that the collation-targettemplate includes a person represented by the face collation template(the original person exists).

Note that the above-described feature quantity and face collation methodused by the collation unit 253 are examples, and other well-knownfeature quantity and face collation method may be used.

The collation unit 253 notifies the output unit 270 of a collationresult. The output unit 270 outputs the collation result acquired fromthe collation unit 253 (step S350). For example, the output unit 270 maydisplay, on a display, a face image, a name, and a collation score of anidentified person.

As described above, according to the second example embodiment, from aface image that is a frame acquired from video data and including aface, the face-feature-point candidate detection unit 231 of the facialauthentication device 200 detects a plurality of face-feature-pointcandidates, by using a plurality of different face-feature-pointdetection methods for a detection-target face feature point. Thereliability degree calculation unit 232 calculates a reliability degreeof the face image, based on statistical information, such as variance,of a plurality of the detected face-feature-point candidates. Based onthe calculated reliability degree, the face image selection unit 233selects a face image suitable for facial authentication.

According to the second example embodiment, by adopting theabove-described configuration, a face image for which a face featurepoint is detected with high accuracy is selected from face images thatare frames constituting video data and each including a face, and theselected face image is used in collation with a face collation template.Thereby, according to the second example embodiment, it is possible toattain an advantageous effect that influence of positional deviation ofa detected face feature point can be suppressed, and highly accuratefacial authentication can be achieved.

Further, for the selected face image, the face-feature-point calculationunit 251 calculates as an integrated face feature point an average ofrespective positions of a plurality of face-feature-point candidates,normalizes the image, by using the integrated face feature point, anduses the normalized image in collation. Thereby, according to the secondexample embodiment, it is possible to attain an advantageous effect thata face feature point can be detected with more accuracy, and facialauthentication can be made with more accuracy.

Note that when the face image selection unit 233 selects a plurality offace images at the processing S240 in FIG. 3 , accuracy of facialauthentication can be further improved by performing the processing S320to S340 in FIG. 8 for each of a plurality of the face images.

FIG. 10 is a diagram illustrating one example of a hardwareconfiguration of a computer device 500 that implements the facialauthentication device of each of the example embodiments. Note that ineach of the example embodiments of the present invention, eachconstituent element of each device represents a block of a functionalunit. Each constituent element of each device can be implemented by anarbitrary combination of software and the computer device 500 asillustrated in FIG. 10 for example.

As illustrated in FIG. 10 , the computer device 500 includes a processor(CPU) 501, a read only memory (ROM) 502, a random access memory (RAM)503, a storage device 505, a drive device 507, a communication interface508, an input-output interface 510, and a bus 511.

The storage device 505 stores a program 504. The drive device 507performs reading and writing to and from a recording medium 506. Thecommunication interface 508 is connected to a network 509. Theinput-output interface 510 outputs and inputs data. The bus 511 connectsthe respective constituent elements to each other.

The processor 501 executes the program 504, by using the RAM 503. Theprogram 504 may be stored in the ROM 502. Alternatively, the program 504may be recorded in the recording medium 506 and is read by the drivedevice 507, or may be transmitted from an external device via thenetwork 509. The communication interface 508 exchanges data with anexternal device via the network 509. The input-output interface 510exchanges data with peripheral devices (such as a keyboard, a mouse, anda display device). The communication interface 508 and the input-outputinterface 510 can function as a means for acquiring or outputting data.Data such as output information may be stored in the storage device 505,or may be included in the program 504.

Note that there are various modified examples of a method forimplementing the facial authentication device. For example, the facialauthentication device can be implemented as a dedicated device. Thefacial authentication device can be implemented by a combination of aplurality of devices.

The template storage unit 254 in the facial authentication device may beimplemented by the storage device 505.

Further, the detection unit 110, the reliability degree calculation unit120, the selection unit 130, the input unit 210, the face-feature-pointcandidate detection unit 231, the reliability degree calculation unit232, the face image selection unit 233, the integratedface-feature-point calculation unit 251, the normalization unit 252, thecollation unit 253, and the output unit 270 in the facial authenticationdevice may be implemented by the processor 501 that performs processingin accordance with program control, for example.

Furthermore, the category of each of the example embodiments includes aprocessing method in which a program for activating the functions so asto be implemented is recorded in the recording medium 506, and theprogram recorded in the recording medium 506 is read as codes and isexecuted in a computer. In other words, the computer-readable recordingmedium 506 is also included in the scope of each of the exampleembodiments. In addition, not only the recording medium 506 in which theabove-described program has been recorded but also the program itselfare included in each of the example embodiments.

The present invention is described above with reference to theabove-described example embodiments. However, the present invention isnot limited to the above-described example embodiments. In other words,according to the present invention, various forms such as variouscombinations and selections of the above-disclosed various elements thatcan be understood by those skilled in the art can be applied within thescope of the present invention.

The present application claims priority based on Japanese PatentApplication No. 2017-124335 filed on Jun. 26, 2017, the entiredisclosure of which is incorporated herein.

REFERENCE SIGNS LIST

-   100 Facial authentication device-   110 Detection unit-   120 Reliability degree calculation unit-   130 Selection unit-   200 Facial authentication device-   210 Input unit-   230 Selection unit-   231 Face-feature-point candidate detection unit-   232 Reliability degree calculation unit-   233 Face image selection unit-   250 Authentication unit-   251 Integrated face-feature-point calculation unit-   252 Normalization unit-   253 Collation unit-   254 Template storage unit-   270 Output unit-   500 Computer device-   501 Processor-   502 ROM-   503 RAM-   504 Program-   505 Storage device-   506 Recording medium-   507 Drive device-   508 Communication interface-   509 Network-   510 Input-output interface-   511 Bus

1. A facial authentication device comprising: at least one memoryconfigured to store instructions; and at least one processor configuredto execute the instructions to perform: detecting, from a plurality offace images each of which including a face of a same person, a pluralityof face-feature-point candidates for at least one face feature point ofthe face, by applying a plurality of different methods to each of theplurality of face images; acquiring variation of positions of theplurality of face-feature-point candidates for each of the plurality offace images; and selecting, from the plurality of face images, one faceimage of which variation of positions of the plurality offace-feature-point candidates is smallest.
 2. The facial authenticationdevice according to claim 1, wherein the at least one processor isconfigured to execute the instructions to perform: calculating, for aface image selected from the plurality of face images, based on theplurality of detected face-feature-point candidates, an integrated facefeature point to be used in authentication of the target face.
 3. Thefacial authentication device according to claim 1, wherein the pluralityof face images are consecutive face images constituting a moving image.4. A facial authentication method comprising: detecting, from aplurality of face images each of which including a face of a sameperson, a plurality of face-feature-point candidates for at least oneface feature point of the face, by applying a plurality of differentmethods to each of the plurality of face images; acquiring variation ofpositions of the plurality of face-feature-point candidates for each ofthe plurality of face images; and selecting, from the plurality of faceimages, one face image of which variation of positions of the pluralityof face-feature-point candidates is smallest.
 5. The facialauthentication method according to claim 4, wherein the facialauthentication method comprises: calculating, for a face image selectedfrom the plurality of face images, based on the plurality of detectedface-feature-point candidates, an integrated face feature point to beused in authentication of the target face.
 6. The facial authenticationmethod according to claim 4, wherein the plurality of face images areconsecutive face images constituting a moving image.
 7. A non-transitoryprogram recording medium recording a program causing a computer toexecute processing of: detecting, from a plurality of face images eachof which including a face of a same person, a plurality offace-feature-point candidates for at least one face feature point of theface, by applying a plurality of different methods to each of theplurality of face images; acquiring variation of positions of theplurality of face-feature-point candidates for each of the plurality offace images; and selecting, from the plurality of face images, one faceimage of which variation of positions of the plurality offace-feature-point candidates is smallest.
 8. The non-transitory programrecording medium according to claim 7, wherein the program recorded inthe non-transitory program recording medium causes the computer toexecute processing of: calculating, for a face image selected from theplurality of face images, based on the plurality of detectedface-feature-point candidates, an integrated face feature point to beused in authentication of the target face.
 9. The non-transitory programrecording medium according to claim 7, wherein the plurality of faceimages are consecutive face images constituting a moving image.